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NASA’s Traffic Aware Planner (TAP) is a cockpit decision support tool that has the 
potential to achieve significant fuel and time savings when it is embedded in the data-rich 
Next Generation Air Transportation System (NextGen) airspace. To address a key step 
towards the operational deployment of TAP and the NASA concept of Traffic Aware 
Strategic Aircrew Requests (TASAR), a system evaluation was conducted in a representative 
flight environment in November, 2013. Numerous challenges were overcome to achieve this 
goal, including the porting of the foundational Autonomous Operations Planner (AOP) 
software from its original simulation-based, avionics-embedded environment to an 
Electronic Flight Bag (EFB) platform. A flight-test aircraft was modified to host the EFB, 
the TAP application, an Automatic Dependent Surveillance Broadcast (ADS-B) processor, 
and a satellite broadband datalink. Nine Evaluation Pilots conducted 26 hours of TAP 
assessments using four route profiles in the complex eastern and north-eastern United States 
airspace. Extensive avionics and video data were collected, supplemented by comprehensive 
inflight and post-flight questionnaires. TAP was verified to function properly in the live 
avionics and ADS-B environment, characterized by recorded data dropouts, latency, and 
ADS-B message fluctuations. Twelve TAP-generated optimization requests were submitted 
to ATC, of which nine were approved, and all of which resulted in fuel and/or time savings. 
Analysis of subjective workload data indicated that pilot interaction with TAP during flight 
operations did not induce additional cognitive loading. Additionally, analyses of post -flight 
questionnaire data showed that the pilots perceived TAP to be useful, understandable, 
intuitive, and easy to use. All program objectives were met, and the next phase of TAP 
development and evaluations with partner airlines is in planning for 2015. 
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ADIF = Aircraft Data Interface Function 
AFS = FAA Flight Standards Service 
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AID 

= 

Aircraft Interface Device 

AOP 

= 

Autonomous Operations Planner 

ASRB 

= 

Airworthiness and Safety Review Board 

ATC 

= 

Air Traffic Control 

ATOL 

= 

Air Traffic Operations Lab - NASA Langley 

ecu 

= 

Communications Convergence Unit 

CDTI 

= 

Cockpit Display of Traffic Information 

DTIF 

= 

Display Traffic Information File 

EDS 

= 

External Data Server 

EFB 

= 

Electronic Flight Bag 

EFIS 

= 

Electronic Flight Instrumentation System 

FDR 

= 

Flight Data Recorder 

FMS 

= 

Flight Management System 

FPM 

= 

Feet Per Minute 

FTOM 

= 

Flight Test Operations Manual 

FTOSR 

= 

Flight Test Operations and Safety Report 

FTRR 

= 

Flight Test Readiness Review 

GAMA 

= 

General Aviation Manufacturers Association 

GPS 

= 

Global Positioning System 

HDU 

= 

High Speed Data Unit 

IFR 

= 

Instrument Flight Rules 

INS 

= 

Inertial Navigation System 

IRB 

= 

Institutional Review Board 

MFD 

= 

Multi Function Display 

MSL 

= 

Mean Sea Level 

NAS 

= 

National Airspace System 

NextGen 

= 

FAA Next Generation Air Transportation System 

OPE 

= 

Operator Performance Laboratory - University of Iowa 

OS 

= 

Operating System 

PF 

= 

Pilot Flying 

PIC 

= 

Pilot in Command 

PM 

= 

Pilot Monitoring 

POI 

= 

Principal Operations Inspector 

STAP 

= 

Simple Text Avionics Protocol (ARINC 834) 

STC 

= 

Supplemental Type Certificate 

SUA 

= 

Special Use Airspace 

SUS 

= 

System Usability Scale 

TAP 

= 

Traffic Aware Planner 

TASAR 

= 

Traffic Aware Strategic Aircrew Requests 

TCAS 

= 

Traffic (Alert and) Collision Avoidance System 

TRL 

= 

Technology Readiness Level 

UAT 

= 

Universal Access Transceiver 

UTAS 

= 

United Technologies Corporation Aerospace Systems 

WAAS 

= 

Wide Area Augmentation System 


I. Introduction 

A utomatic Dependent Surveillance-Broadcast (ADS-B) technology is a cornerstone of the Federal Aviation 
Administration’s Next Generation Air Transportation System (NextGen), 1 and ADS-B Out (transmission) 
capability has been mandated in the United States (U.S.) for installation in aircraft by 2020. In order to help aircraft 
operators achieve near-term operational benefits from ADS-B, NASA developed the Traffic Aware Strategic 
Aircrew Requests (TASAR) concept 2 , which uses ADS-B In (reception) data from other aircraft and ground stations 
to provide pilot advisories of traffic -compatible lateral and vertical trajectory changes that would be beneficial to the 
flight. To achieve operational deployment of TASAR, NASA developed the Traffic Aware Planner (TAP) 3 , a 
cockpit-based software application that enables TASAR operations. A flight trial program was conducted to 
evaluate TAP in a representative airborne environment. This paper addresses the findings from that flight trial. 
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A. Flight Trial Rationale 

The primary purpose for flight-testing TAP was to bridge the critical divide between Technology Readiness 
Level (TRL) 4 (testing in a laboratory environment) and TRL 6 (testing in a relevant environment), as a stepping- 
stone to full operational deployment (TRL 9). Since the TASAR concept is intended to provide near-term 
operational benefits, NASA has undertaken a number of activities to accelerate its adoption by the operator 
community: NASA has analyzed the potential user benefits 3 4 , the operational hazards 5 , and the Federal Aviation 
Administration (FAA) certification and operational approval requirements 5 for TASAR. NASA has developed the 
TAP software application and has evaluated the human-machine interface (HMI) in a high-fidelity fixed-base flight 
simulator during a human-in-the-loop experiment with active airline pilots 6 . To further prepare TAP for operational 
readiness, a flight trial was conducted to assess the software in a representative airborne environment. 

TAP’s architecture and algorithms were derived from the NASA Autonomous Operations Planner (AOP) 7 , a 
self-separation research prototype system developed for simulation-based experiments in NASA Langley’s Air 
Traffic Operations Laboratory (ATOL). The flight-evaluation program entailed the migration of TAP from a 
simulation environment to a live-avionics airborne operational environment, so that factors unique to the flight 
environment could be assessed. This paper addresses the culminating phase of the 18-month development program, 
the TAP flight evaluations. 

B. TASAR Benefits Analysis 

NASA developed TASAR as a cockpit-based automation capability that enables aircraft operators to benefit 
from real-time advisories of traffic -compatible lateral and vertical trajectory changes that save flight time and/or 
fuel. A preliminary benefits analysis 4 indicated that TAP-equipped aircraft could achieve average time savings of 
approximately one to four minutes per operation, and fuel savings between 50 to 550 lbs. per operation depending 
on the objective of the aircrew, class of airspace user, and aircraft type. Benefits should increase with longer stage 
lengths, since beneficial trajectory changes can be applied over a longer distance. The onboard automation leverages 
ADS-B surveillance information to increase the likelihood of ATC approval of pilot-initiated trajectory change 
requests, thereby increasing the portion of the flight flown on or near a desired trajectory. All automation and pilot 
procedures are fully dedicated to a single aircraft that allows tailoring of the optimization criteria to the specific 
objectives of each flight, and provides for timely responses to changing situations. TAP can use traffic and weather 
information derived from multiple sources, including ADS-B and internet-based data available through broadband 
Internet connectivity, to de-conflict its optimized flight path recommendations from other traffic. Special Use 
Airspace (SUA), and significant weather systems. This capability should maximize the chances of an aircrew- 
initiated rerouting request being accepted by Air Traffic Control (ATC), increasing the flight time spent on the 
desired trajectory, and reducing radio transmissions and the associated frequency congestion. 

C. TAP Functionality 

The target user group for TAP comprises Flight Management System (FMS) -equipped civil aircraft operating in 
the continental U.S. under Instrument Flight Rules (IFR). The TAP software application is designed to run on a 
Class 2 EFB computing platform installed on the aircraft's flight deck, with data connections to aircraft avionics and 
the inflight Internet. In its automatic mode, TAP monitors the aircraft's trajectory and operating environment (e.g., 
weather information) and periodically scans for lateral and vertical changes that produce time or fuel savings (Figure 
1). The solutions are displayed in textual form to the flight crew and in a “selected optimization’’ graphical form 
(Figure 2), for the crew’s optional use in requesting in-flight trajectory changes from ATC. 
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Figure 1. TAP Auto mode - main screen. 


Figure 2. TAP Auto mode: selected optimization. 


TAP leverages ADS-B In information to check trajectory changes for potential traffic conflicts, filtering these 
out to improve likelihood of ATC approval and thereby increasing user benefits. TAP also enables pilots to 
manually enter potential trajectory changes for evaluation by the TAP software (Figures 3 and 4). 



Figure 3. TAP Manual mode: data entry interface. 
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Figure 4. TAP Manual mode: evaluation screen 


D. TASAR Certification and Operational Approval Assessment 

One of the primary objectives of the TASAR project was to identify and address the certification and operational 
approval considerations for the eventual operational deployment of TAP. As part of a detailed regulatory analysis 4 5 , 
preliminary discussions were held with representatives from the FAA Aircraft Certification Service, Systems & 
Equipment Standards Branch (AIR- 130) and the Flight Standards Service, Flight Technology Requirements Branch 
(AFS-430) to determine the likely design assurance levels for the TAP application. TAP was categorized as an 
ADS-B enabled EFB Type B 8 performance/planning software application, rather than a dedicated ADS-B In 
application. On this basis, a Minor failure effect classification was established for the flight trial, which will also 
apply to an operationally deployed system, subject to a prohibition of the display of ownship position in flight. From 
these preliminary discussions, it appears that the TAP software will not require RTCA DO-178B software 
certification, and the only necessary airworthiness approvals will be for the EFB mounting and data interface 
provisions. The TASAR safety and hazard evaluations were based on an analysis of the TASAR Concept of 
Operations 9 and associated use cases, and a determination of the attendant operational hazards. 10 These analyses 
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confirmed that the use of TAP represents a Minor risk, as defined by FAA AC23.1309-1E, 1 * * * * * * * * * 11 both for the flight trial 
and for subsequent deployments to potential end-users. 

Customer operational approvals will be determined by the Carrier’s Principal Operations Inspector (POI), using 
the established methodology for the issuance of Operational Specification A061 Use of Electronic Flight Bag. 
Collectively, these requirements will impose minimal additional burden on prospective TAP users, if they already 
have airworthiness and operational approvals for the use of EFB equipment in their fleets. 

In order to accelerate the operational deployment of TAP, an early decision was made for the flight trial to avoid 
the use of an Experimental Category flight permit, and to progress directly to a flight evaluation on an aircraft with a 
Normal category Certificate of Airworthiness operating under IFR in the National Airspace System (NAS). With 
this approach, all of the technical, operational, safety, and certification considerations related to the flight trial will 
be reusable for subsequent deployments of the system. Airworthiness approvals for the hardware 12 and software 13 
were obtained by the issuance of Supplemental Type Certificates (STC) for all TAP -related equipment. This 
achieved the dual goals of hosting TAP in a representative flight environment, while avoiding the geographic, 
weather, and essential-crew-only restrictions associated with experimental flight operations. 

II. Flight Trial Design 


A. Flight Trial Objectives 

One of NASA’s primary objectives for the TASAR program is to reduce obstacles for users to achieve near-term 
direct benefits of ADS-B. 14 A crucial step towards achieving this outcome is the successful in-flight validation of 
TASAR functionality in the National Airspace System (NAS), using the TAP software platform and the ambient 
live-avionics data infrastructure. This was the primary goal of the flight trial, with a secondary goal of reducing the 
risks for subsequent operational implementations of TAP. In an effort to achieve these goals, the following 
incremental objectives were defined: 

1) Verification of the TAP data interfaces. 

2) Verification of the TAP software functionality. 

3) Pilot assessment of TAP usability in an operational environment. 

4) Opportunity-based TAP optimization requests to ATC. 

B. Test Plan 

A 10-flight 30-hour flight-evaluation program was planned, including four hours of shakedown testing and 26 
hours of dedicated TAP evaluations. An incremental flight test approach was used to minimize the program risks 
associated with the porting of the complex TAP software to a new hardware platform in the highly regulated 
environment of an aircraft with a Normal Category Certificate of Airworthiness. Avoiding the use of an 
Experimental Flight Permit will help accelerate TAP’s transition from the simulation laboratory to the NAS by 
forcing a substantially higher level of maturity on the system and its integration with the aircraft. Key steps towards 
achieving this goal included rapid -prototyping usability studies with interactive HMI mockups, followed by 
comprehensive evaluations in a high-fidelity flight simulator. These evaluations, conducted in the University of 
Iowa’s Operator Performance Lab (OPL), were used to refine the TAP HMI, test the pilot procedures developed 
from the TAP use cases, and assess pilot workload and situation awareness impacts under nominal and off-nominal 
conditions while using TAP. The flight trial test plan was developed in parallel with that for the OPL evaluations, 
and the flight trial followed the OPL data collection within three months. The test plan detailed these activities, as 
well as the flight profiles, test matrix. Evaluation Pilot preparations, and flight procedures for the flight evaluations 
that followed the OPL trials, as discussed below. 

1. Flight Profiles 

The trials were conducted in the NAS along the U.S. eastern seaboard, with all flights originating and 

terminating at the Newport News / Williamsburg International Airport (KPHF). A number of “round-robin” (out and 

back) candidate routes were developed in the test plan, based on published high-altitude IFR navigation aids and 

fixes. Four flight profiles were selected to exercise TAP’s route optimization functionality in the presence of 

potential SUA and traffic conflicts (Figure 5 - Figure 7, screen shots from Jeppesen Mobile FliteDeck). Wind-field 

data derived from live broadband connectivity to the National Oceanographic and Atmospheric Administration 

(NOAA) Rapid Refresh system were incorporated in the TAP optimizations. The evaluations proceeded in four 

increasingly challenging steps, starting with the establishment of basic data connectivity, proceeding through system 

functionality verifications, and culminating in the operational use of TAP to generate trajectory optimizations. 
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Figure 5. TAP Flight Trial, Route 1. 



Figure 6. TAP Flight Trial, Route 3 
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Figure 7. TAP Flight Trial, Routes 5 and 6. 


2. Test Matrix 

Table 1 shows the test matrix for the planned evaluation flights. Initial cruising altitudes were as shown in the 
table, but the TAP optimizations led to final altitudes ranging from 17,000 ft. MSL to FL370. Weather avoidance 
was not evaluated in the initial flight trials, and SUA boundaries were hard-coded into a database available to TAP 
and treated as “permanently hot” airspace. As shown in the table, the routes were designed to control the impact of 
traffic and SUA on the route optimization process. Due to ATC workload, the final three flights initially planned for 
Route 6 (low altitude) were replaced with Route 5 (high altitude). The final two flights also used a slightly modified 
version of Route 5 to eliminate the overlap of outbound and inbound legs. 
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Table 1. Flight Evaluation Matrix. 


Flight # 

Route # 

Direction 

Initial 

Flight Fevel (FF) 

Traffic Density 

SUA Proximity 

1 

1 

ccw a 

FL300 

Low 

No 

2 

Flight aborted 6 

- 

- 

- 

- 

3 

3 

ccw c 

16,000 ft. 

Low 

Yes 

4 

3 

cw 

16,000 ft. 

Low 

Yes 

5 

5 

ccw 

FL340 

High 

No 

6 

5 

cw 

FL340 

High 

No 

7 

6 

ccw 

FL210 

High 

Yes 

8 

5 d 

cw 

FL310 

High 

No 

9 

5 d,e 

ccw 

FL310 

High 

No 

10 

Cl /--! 1 1 

5 d,e 

b^i- i . i . i i 

cw 

FL310 

High 

No 


“Clockwise. b Flight aborted due to an aircraft equipment malfunction unrelated to TAP. 'Counterclockwise. 
d Originally planned as Route 6. e Included minor revision to route to eliminate overlap. 


3. Flight Crew 

The flight crew consisted of a Pilot in Command (PIC) / Safety Pilot and an Evaluation Pilot. Nine senior IFR- 
rated Evaluation Pilots were employed in the study, consisting of six senior airline captains, one senior first officer, 
and two highly experienced aviators. Operator communities represented included major, regional, and low-cost 
airlines, and high-end General Aviation. The NASA TASAR team Evaluation Pilots conducted the end-to-end 
systems checkouts and procedure rehearsals on flights 1 & 2, and the remaining flights were performed with the 
eight independent external Evaluation Pilots. The Evaluation Pilots were mailed comprehensive briefing packages 
one week before their flight trial. They received approximately two hours of dedicated TAP training on the day of 
their flight, followed by comprehensive pre-flight mission and aircraft safety briefings. Each Evaluation Pilot was 
thoroughly debriefed post-flight. 

The sole function of the Safety Pilot was to assure safety of flight. The Safety Pilot played no role in the 
Evaluation Pilot’s use of TAP, except for providing assistance in the configuration of the aircraft’s displays and 
interacting with the FMS as requested. The Safety Pilot performed all normal aircraft operations during the 
outbound legs of each flight, but assigned radio communications duties to the Evaluation Pilot during the inbound 
legs in order to provide representative Pilot Monitoring (PM) workload for this phase. 

The cabin crew typically comprised five personnel: a Test Director, a Flight Test Engineer, one or two TAP 
Software Engineers, and one or two NASA Researchers. The Test Director orchestrated and performed all aspects of 
evaluation-pilot data collection during the flights and was the conduit for all TAP -related internal aircraft 
communications with the Evaluation Pilot and Safety Pilot. The Test Director was also responsible for the optimized 
sequencing and conduct of all test points. The aircraft Flight Test Engineer was responsible for the aircraft’s data 
systems and TAP interfaces. The TAP Software Engineers exercised the software independently from the Evaluation 
Pilot, and performed any necessary real-time troubleshooting. The NASA Researchers also operated independent 
instances of the TAP application for real-time monitoring and testing. 
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C. Data Collection 


1. Flight Procedure 

Table 2 shows the planned sequence of events for each test flight. The Safety Pilot performed the takeoff and 
climb-out to 10,000 ft., at which time the Evaluation Pilot unstowed the EFB and mounted it on the windshield. The 
Test Director then orchestrated a highly scripted scenario that guided the Evaluation Pilot to methodically evaluate 
TAP’s features and controls. Six phases of TAP usage were evaluated: two involving the startup and shutdown of 
the TAP EFB, two using the TAP Auto mode, and two using the Manual mode. As shown in the table, the 
Evaluation Pilot concentrated solely on TAP operations on the outbound legs and transitioned to additional PM 
duties on the inbound segments. The latter entailed responsibility for ATC communications, weather avoidance, and 
the planning and execution of a TAP optimization maneuver. The Evaluation Pilot interacted with both the TAP 
Auto and Manual modes during the outbound and inbound legs, and each Auto/Manual mode segment was followed 
by the administration of an in-flight questionnaire including the Bedford Workload Scale 13 . 

Table 2. Evaluation flight timeline and phases. 


Flight 

Leg 

Procedure and 
Flight Phase 

Eval. Pilot 
Role 

ATC 

Request? 

Duration 

(min.) 

Tot. Elapsed 
Time (hh.min.) 

On ground 

Power up EFB, initialize TAP 

TAP only 

- 

5 

5 

Outbound 

Departure and climb through 10,000 ft. 

- 

- 

10 

15 

Outbound 

Phase 1 : EFB unstow, mount 

TAP only 

- 

5 

20 

Outbound 

Phase 2: Auto mode scenarios 

TAP only 

No 

20 

40 

Outbound 

Phase 3: Manual mode scenarios 

TAP only 

No 

20 

1:00 

Transition 

Reconfigure for inbound leg 

- 

- 

10 

1:10 

Inbound 

Phase 4: Auto mode scenarios 

PM a 

Yes 

20 

1:30 

Inbound 

Phase 5: Manual mode scenarios 

PM 

Yes 

20 

1:50 

Inbound 

Phase 6: TAP shutdown 

TAP Only 

- 

10 

2:00 

Inbound 

EFB stowage (approx. 10,000 ft.) 

PM 

- 

5 

2:05 

Inbound 

Descent and landing 

- 

- 

15 

2:20 


“Pilot Monitoring duties 


D. Test Platform 

The following paragraphs address the selection, modification, and certification of the flight test platform, its 
ADS-B installation, and the broadband system installed in the aircraft. 

1. Aircraft 

A Piaggio PI 80 Avanti, operated by Advanced Aerospace Solutions ( Ad v Aero), served as the test aircraft for 
the flight trial. Four characteristics of this test platform directly supported the objective to accelerate the operational 
readiness of TAP: the aircraft has a Normal Category airworthiness category; it is certified for single -pilot 
operations; it has a large cabin; and it has a broad flight envelope. Retention of a Normal Category Certificate of 
Airworthiness will significantly streamline the reuse of the TAP software for subsequent deployments of the system. 
Avoidance of an Experimental Category classification was also important for the reasons listed in the flight trial 
rational section above. The single-pilot certification eliminates restrictions regarding the occupancy of the copilot’s 
seat, and it also enables the use of uncertified software on the non-handling side of the cockpit. The Avanti ’s large 
cabin accommodated a seven-person test-crew comprising: a Safety Pilot, an Evaluation Pilot, the Test Director, a 
Flight Test Engineer, a Data Engineer, a TAP Software Engineer, and a NASA Researcher. A broad flight envelope 
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was deemed essential to achieving the objective of hosting the TAP prototype in a representative operational 
environment. The Avanti is certified for all-weather single-pilot operations. It has a cruise speed of approximately 
375 knots (0.65M) at 28,000 ft. and a ceiling of 41,000 ft. The aircraft is very fuel-efficient, allowing for longer test 
missions, and the cabin is large enough to accommodate the full test crew with their laptops and recording 
equipment. 

2. Traffic Processor 

To ensure the transferability of the TAP installation to future operational platforms, a field-proven hardware 
design was adopted for the flight trials, comprising a third-generation ACSS TCAS 3000SP ADS-B In/Out capable 
Traffic Collision Alerting System (TCAS), paired with a United Technologies Corporation Aerospace Systems 
(UTAS - formerly Goodrich) SmartDisplay™ Class 2 EFB 16 . The high cruising altitude of the Avanti excluded the 
use of Universal Access Transceiver (UAT) technology for the ADS-B installation, because UAT is constrained to 
altitudes below 18,000 feet. Accordingly, a 1090 MHz extended squitter (1090ES) solution was employed. The 
latter is also the standard adopted outside the U.S. and is therefore more suitable for downstream installations for 
operators engaged in international operations. 

3. Aircraft Interface Device 

A prerequisite for the successful TAP deployment to the user community will be its ease of installation. TAP 
must interface to a number of aircraft data sources, which could vary widely from aircraft to aircraft. A decision was 
made to standardize all TAP-to-aircraft interfaces using a certified Aircraft Interface Device (AID) that embodies all 
of the required connectivity. Future TAP installations based on the industry standard Simple Text Avionics Protocol 
(STAP) feed, as provided by the AID, will substantially reduce the technical and certification risks, while facilitating 
the configuration management task across fleets. A compact UTAS AID was chosen for the flight trial. The unit 
incorporates ten ARINC 429 input channels, six 10/100 Base-T ports, and eight discrete inputs. It connects to one or 
two EFBs using an ARINC 828 EFB standard interface 17 . The AID’S avionics bus interfaces include: primary 
Global Positioning System (GPS) with Wide Area Augmentation System (WAAS), Inertial Navigation System 
(INS), Flight Management System (FMS), Air Data Computer (ADC), and the Flight Data Recorder (FDR) bus. The 
AID packages the incoming ARINC 429 data using STAP, as defined by the ARINC 834 Aircraft Data Interface 
Function (ADIF) standard 18 , for onward transmission to the EFBs and other research processors connected via the 
local area network. Use of these industry-standard protocols will minimize the technical risk for future TAP 
installations because the TAP software remains agnostic to the aircraft interfaces upstream of the AID. In addition, 
this approach will allow for low-cost, one-time, data-concentrator installation that will remain permanently installed 
in the target aircraft, minimizing recurring expenses and downtime associated with continual installation and 
removal of the system. 

4. EFB 

The UTAS SmartDisplay™ G500 EFB was 
selected for the flight trials (Figure 8). For risk 
mitigation purposes, a requirement for the chosen 
system was for it to support a certified operating 
system (OS), should it eventually be required in 
future testing. For example, if a Cockpit Display 
of Traffic Information (CDTI) with ownship 
position were found necessary for TAP 
operation, this would entail the use of a certified 
OS because CDTI requires a certified platform. 

Physical size and processor performance 
requirements were the primary considerations 
leading to the selection of the SmartDisplay, 
which also supports an alternate-boot option for 
the DEOS™ operating system and therefore 
meets the certification requirement. The 
SmartDisplay™ is a tablet EFB with a 1024 x 
768 pixel resistive touch screen. The unit 
incorporates an Intel Core Duo 1.66 GHz 
processor with 2GB RAM and 16 GB of 
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removable memory on a solid-state compact flash card. The TAP application was hosted locally on the 
SmartDisplay, which communicated with all the aircraft data systems via the AID. The Evaluation Pilot’s EFB was 
mounted on the copilot’s right-side windshield using dual suction cups, at a location of the pilot’s choosing. 

Additional instances of TAP were hosted on a second SmartDisplay™ EFB and two engineering laptops 
available to crewmembers in the aft cabin, for independent TAP evaluations and debugging purposes. A final 
instance of TAP was installed on each of the test-bed’s two flight-test computers, primarily for gathering data for 
post-flight analysis pertaining to aircraft integration issues. 

5. Broadband Airborne Internet 

One of the objectives of the flight trials was to integrate broadband Internet-sourced wind-field data into TAP’s 
trajectory processing and optimization functions. A certified, single-channel, Inmarsat broadband link was 
incorporated to allow TAP to access real-time, hourly updates of such information. Principal system components 
included a High Speed Data Unit (HDU) Transceiver, a Communications Convergence Unit (CCU), and a power 
amplifier. The system bandwidth was limited to 200 kbps, due to size constraints of the installed low profile low- 
gain blade antenna. The CCU acted as a full-service router for the aircraft, with eight LAN-10/100 ports and Wi-Fi® 
connectivity for a maximum of 54 concurrent clients. The CCU enabled the engineers in the cabin to run TAP on an 
additional EFB and several laptops, independent of the Evaluation Pilot’s EFB, using wireless access to the AID 
data. This feature proved important for independent data logging and real-time software troubleshooting during the 
trials. An External Data Server (EDS) application was developed to allow TAP clients to access the wind data via an 
Internet connection to the NOAA website. The EDS will support possible expanded data sources for future trials, 
such as convective weather hazards, dynamically updated SUA boundaries, and supplemental traffic state and/or 
intent data not currently available via the ADS-B In channel. 

E. Flight Test Data 

Flight test data were collected using a combination of in-flight hand-recorded questionnaires, digital and video 
data, and post-flight questionnaires and debriefings. 

1. In-Flight Questionnaires 

The Test Director administered a questionnaire to the Evaluation Pilots during each of the two outbound and two 
inbound flight legs, following each scripted evaluation of TAP operating modes. This questionnaire addressed items 
such as TAP trajectory-optimization solutions, time/fuel outcomes, traffic/SUA impacts, anomalous behaviors, and 
included scenario-specific probes. Additionally, each Evaluation Pilot completed the Bedford Workload Scale, a 
uni-dimensional rating scale designed to assess the pilot’s spare mental capacity while completing a task. The single 
dimension is assessed using a hierarchical decision tree that guides the pilot through a ten-point rating scale, each 
point of which is accompanied by a descriptor of the associated level of workload. All questionnaire data were hand- 
recorded in flight by the Evaluation Pilot and entered into a database after landing for post-processing and analysis. 

2. Digital & Video Data 

The ten ARINC 429 data-busses interfaced to the AID were automatically recorded during each flight. These 
busses included the FMS flight plan data and the ADS-B In information used by TAP. Each TAP client also 
generated a complete log file that was downloaded after each flight. The recorded aircraft data had sufficient fidelity 
and extent to support real-time playback of the entire flight using AdvAero’s engineering simulator. In addition, 
digital (de -identified) cockpit video was captured showing the Evaluation Pilot’s interactions with TAP for post- 
flight analysis and ambiguity resolution. 

3. Post-Flight Questionnaire and Debriefings 

Following their flight, the Evaluation Pilots completed a comprehensive online post-flight debriefing 
questionnaire, including the System Usability Scale (SUS), which took approximately an hour to complete. The SUS 
is a short, 10-item Likert scale designed to measure a user’s perceptions of system usability. SUS yields a single 
number representing a composite measure of the overall usability of the system being studied. Note that scores for 
individual statements are not meant to be interpreted individually as is prescribed by Brooke 19 . The SUS score is 
calculated by first summing the score contributions of each item which range from 0 to 4 for all statements. For 
statements 1, 3, 5, 7, and 9, the score contribution was the scale position minus 1. For statements 2, 4, 6, 8, and 10, 
the contribution was 5 minus the scale position. The final SUS score was calculated by multiplying the sum of the 
scores by 2.5 to obtain the overall value of system usability. SUS scores had a range of 0 to 100. 
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After each questionnaire had been scored, the Evaluation Pilot, Test Director, TAP Engineer, NASA 
Researcher, and Safety Pilot participated in a group debriefing to review all of the in-flight and post-flight 
questionnaires and to capture any lessons learned for the next flight. The questions addressed five evaluation 
categories: TAP Operational Usage, TAP Performance, TAP HMI, TAP Benefits, and an overall assessment of the 
test program. This final debriefing typically lasted two-hours and was used to consolidate and clarify all pertinent 
information gained from each flight -test, including the questionnaire findings. 

III. Flight Trial Results 

The results of the data collected during nine flights are presented in the following section. One flight was 
curtailed just after takeoff because of an aircraft unserviceability unrelated to TAP. The results are presented in four 
sections, corresponding to the four flight trial objectives listed in Section II. A: 1) TAP Data Interface Verification; 
2) TAP Functionality Verification; 3) Subjective Measures; and 4) TAP Operational Use. 

A. TAP Data Interface Verification 

The initial phases of the TAP integration into the test platform posed a number of challenges related to the 
aircraft-side data sources. Chief among these was the General Aviation Manufacturers Association (GAMA) 
General Aviation A429 subset output by the Avanti FMS, which differed from the enhanced A429 structure used in 
the NASA ATOL. GAMA data are usually used for the display of route and waypoint symbology on Electronic 
Flight Instrumentation System (EFIS), Multi Function Display (MFD), or weather radar displays. These data had to 
be supplemented by other sources, such as the FDR bus, to meet TAP data requirements. 

The flight-trial version of the TAP software was developed in iteration with the aircraft integration activity. 
Three data-related problems hampered the pre-test integration efforts on the ground. TAP required the aircraft to 
provide valid air data in order to function, which entailed the use of an unwieldy ADC test set for all ground tests to 
stimulate an in-flight condition. Similarly, a complex procedure was required to force the Inertial Navigation 
Systems into a simulation mode that would allow the aircraft to generate usable groundspeed data for TAP. In 
addition, limited satellite signal reception in the airport environment compromised EDS connectivity during ground 
testing. Due to these issues, the integration logistics were more challenging than originally anticipated. Advantage 
was taken of additional data collection tools in the TAP software, FMS, and aircraft computer servers that enabled 
the unattended collection of TAP data while the aircraft was engaged in flights unrelated to TAP. These data could 
then be fed through a playback capability of the TAP software to conduct integration testing. Almost 40 hours of 
these in-flight opportunity data were collected, reducing the dedicated TAP -integration flight time requirement to 
3.2 hours. After the troubleshooting was completed, a successful end-to-end system test was performed immediately 
prior to the shakedown flight. The TAP data interfaces functioned satisfactorily thereafter, allowing TAP to perform 
as designed and to successfully download the updated wind data to the EDS. Nevertheless, a number of data 
problems persisted during the flight trials, including data dropouts, latency, and vertical speed noise. These artifacts 
are inherent to a live avionics and ADS-B environment, though some were artifacts of the flight testing, as described 
in the following sections. 

1. Data Dropouts and Latency 

The AID was designed to serve only two clients: a pilot EFB and a copilot EFB. Accordingly, the AID may have 
been overloaded with up to five EFBs and laptops in the flight -test aircraft subscribed to AID data. AID loading is a 
function of the number of ARINC data labels (items) per message, the message rate, and the number of clients. The 
limiting case was found to be the transfer of a full flight plan, which required multiple messages from the FMS to 
fully define the route. This was the only data label with inter-arrival times that exceeded one second for a significant 
percentage of time (96%). When thus overloaded, the AID exhibited an increase in dropped messages. Figure 9 
contrasts the differing data dropouts for the receipt of a full flight plan on the Evaluation Pilot’s EFB, the TAP 
engineer’s laptop, and the second EFB in the aft cabin (dropouts are not shown for up to two additional laptops in 
the aft cabin). Longer times between receipt of a full flight plan are due to data dropouts. The approximately 27- 
second dropout during Flight 4 coincided with an additional laptop (not shown in Figure 9) subscribing to the AID. 
This resulted in dropped ARINC data labels from the FMS and GPS, but not from the INS, ADC, flight control 
computer, or TCAS. The engineering laptop experienced more dropouts than the EFBs across all flights, even 
though the highest dropout time duration was experienced on the Evaluation Pilot’s EFB during Flight 4. The 
manufacturer of the AID indicated that AID data dropout behavior is unpredictable when overloaded, and it may not 
always be the case that laptops experience higher dropouts than EFBs. 
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None of the flight plan dropouts impacted the flight-test TAP advisories, since these dropouts occurred at times 
when the flight plan was not changing. Furthermore, the AID loading of up to five devices is not representative of 
typical operations, where one or two EFBs subscribe to AID data. Nevertheless, flight plan dropouts have been 
identified as a potential area of concern, and future work may be needed to determine how often these dropouts 
coincide with flight plan changes that could result in TAP optimizing against an out-of-date flight plan. 
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Figure 9. Maximum observed time between TAP receiving full flight plan. 


2. Vertical Speed Noise 

Examination of vertical speed data for both for the ownship and ADS-B traffic indicated vertical speed variations of 
up to +4000 feet per minute (FPM). This phenomenon was particularly noticeable on Flight 4, which experienced 
the largest altitude rate change for the ownship during a 5-second window of clear air turbulence beginning at 
approximately 4200 seconds, as shown in Figure 10. The 27-second data dropout experienced during Flight 4 
occurred about 50 minutes before this turbulence, and did not impact the vertical speed coming from the ah' data 
system. 
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Figure 10. Turbulence effects on ownship altitude-rate. 
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TAP does not use vertical speed for ownship predictions, but there is a concern about the stability of traffic 
ADS-B vertical speed values and the impact they may have on traffic trajectory prediction. Vertical speed is used by 
TAP to predict the vertical component of the ADS-B traffic trajectory since the target altitude is not part of the 
ADS-B message set received by TAP. In a simulation environment, the vertical speed value is generally stable and 
accurate. In flight, turbulence may result in brief accelerations that, in turn, can result in temporarily large vertical 
speed being broadcast. 

The vertical speed noise presented in this subsection could potentially cause TAP to generate advisories that are 
traffic-incompatible and/or less beneficial, if not otherwise mitigated. More work is needed to quantify how often 
vertical speed noise impacts TAP advisories and to determine if methods, such as filtering algorithms, could 
increase TAP performance. No significant outliers or noise was observed for any other ARINC data labels across the 
nine flights. 

B. TAP Functionality 

In almost every case, the TAP software passed the full set of initialization-sequence connection tests, entered the 
online mode at 10,000 ft. as designed, and generally operated stably throughout the flight. Two aspects of the 
software caused occasional problems during the trials: the aircraft performance model and the ADS-B vertical speed 
fluctuations discussed above. (The TAP optimization outcomes are discussed separately in Section D below). 

TAP trajectory generation and optimizations rely on an accurate digital aircraft performance model, which was 
unavailable for the Avanti test-bed aircraft. Development of such digital models from aircraft performance 
handbooks is time-consuming and expensive, and the magnitude of the task was underestimated. As a compromise, 
a performance model of a generic twin-engine, medium-sized jet was scaled to approximate the en-route 
performance of the Avanti (a high-performance turboprop). This approximation impacted TAP’s predictive 
capability and curtailed all route optimization computations while the aircraft was climbing or descending. Use of 
this model also introduced errors in the fuel and time calculations that made precise quantification of the 
optimizations difficult. An accurate Avanti model is in development for the next phase of flight trials, and a new 
trajectory generator is being developed that will be more flexible to variations in available performance data for 
other aircraft types that will be used in future trials or TAP operational deployments. 

C. Subjective Measures 

In an effort to assess pilot workload and usability, and to better understand how the Evaluation Pilots perceived 
their interaction with the TAP software application in terms of intuitiveness, understandability, and usefulness, 
several subjective measures were administered both during and after their flight. 

Generally, the Evaluation Pilots rated usability of TAP as high, and their perceived workload was low. In the 
post-flight questionnaire and debrief, the Evaluation Pilots reported that operating the TAP software application was 
relatively easy and that the formal training provided pre-flight was adequate. The following are examples of 
comments from the Evaluation Pilots during the post-flight debriefing: 

"Given the time we had to prepare - I felt comfortable using TAP in the airplane with no prior real-life 
experience, ” ‘‘Very well thought out implementation, ” and ‘‘Intuitive and easy to understand. ” 

The results of the analyses of the subjective measures administered during the flight trial are presented below 
under the following categories: Bedford Workload Scale, System Usability Scale (SUS), Post-Flight Questionnaire, 
and TAP enhancement requests. 

1. Bedford Workload Scale 

The average Bedford Workload Scale ratings for each segment of the flight are depicted in Figure 11. Generally, 
across all flight segments including all TAP Mode Conditions (Auto Mode, Auto Mode Pilot Monitoring, Manual 
Mode, Manual Mode Pilot Monitoring), the Evaluation Pilots’ reported ratings of perceived workload were very 
low. 

There was a small increase in average workload between Auto and Manual Modes, and also between no-PM and 
PM conditions. Because TAP Manual Mode inherently involves more interaction from the pilots as they are entering 
their own trajectories, this increase in perceived workload was not unexpected. When the pilots assumed the 
additional PM duties, the overall workload in the cockpit increased. Consequently, higher ratings of perceived 
workload were expected in that condition. However, it is important to note that, as depicted in Figure 11, the 
Evaluation Pilots’ ratings of perceived workload were almost all below three. According to the descriptors on the 
Bedford Scale, shown in Figure 12, ratings below three indicate that the pilots typically believed that workload was 
tolerable for the task, and they had enough spare mental capacity for all desirable additional tasks during flight 
operations. 
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Figure 11. Average Bedford Workload Scale ratings for TAP Mode Conditions. 


Bedford Workload Scale 



OPERATOR DEMAND LEVEL 


RATING 


1 Workload Insignificant. 

in 

1 Workload low. 


Enough spare capacity for all desirable additional tasks. 




1 Insufficient spare capacity for easy attention to additional 
1 tasks. 

* 1 

4 Reduced spare capacity. Additional tasks cannot be given 
1 the desired amount of attention. 

5 j 

Little spare capacity. Level of effort allows little 
attention to additional tasks. 

6 j 



1 Very little spare capacity, but the maintenance of effort 

Q 

1 Very high workload with almost no spare capacity. 
1 Difficulty in maintaining level of effort. 

e 



Extremely high workload, no spare capacity. Serious 
doubts as to the abilitv to maintain level of support. 

li 



| Tasks abandoned. Pilot unable to apply sufficient effort. 



Enter Here 


(Roscoe, 1984) 


Figure 12. Bedford Workload Scale. 
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2. System Usability Scale 

The Evaluation Pilots were administered the System Usability Scale (SUS) 19 post-flight, and as can be seen in 
Figure 13, the pilots rated the usability of TAP as high (M = 81.3, SD = 14.8). This result is consistent with the 
results from the SUS data collected during the HITL-I experiment conducted in the Operator Performance 
Laboratory (OPL) at the University of Iowa 6 . During this experiment, the participant pilots were administered the 
SUS after each use case and the data was collapsed across use cases for Manual (M = 87.36, SD = 12.23) and Auto 
Modes (M = 87.08, SD = 12.46). As Figure 13 illustrates, the mean SUS scores from the flight trial and simulation 
experiment were consistent indicating that Evaluation Pilots perceived TAP to be easy to use in both environments. 
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Figure 13. System Usability Scale (SUS) calculated scores. 

3. Post-Flight Questionnaire 

In addition to the SUS, an online post-flight questionnaire was administered to all Evaluation Pilots after 
completion of their flight. The questions can be grouped into three categories: a) overall understandability/usability 
findings, (b) specific HMI design changes, and (c) general feelings about TASAR’s utility and effect. The results for 
each category of questions are presented in this section. 

a) Overall Understandability/Usability Findings 

Figure 14 depicts the results of selected questions in the post-flight questionnaire that attempted to gain an 
understanding of TAP usability and the extent to which the pilots were able to understand the interface. For 
example, “How difficult or easy was it to use the overall Auto Mode interface?’’ Specifically, these questions 
attempted to assess how well the pilots were able to use and understand the overall Auto and Manual Modes. The 
pilots responses to these questions were collected via a 7-point Likert scale with the anchors 7 = Very Easy and 1 = 
Very Difficult. Generally, the Evaluation Pilots rated the TAP interface, in both Auto and Manual Modes, as very 
easy to use and understand, with average scores on the Likert scale all above 5. 

b) Specific HMI Design Changes 

Figure 15 depicts the results of selected questions that assessed the extent to which it was easy or difficult to 
use specific HMI design characteristics. For example, “How difficult or easy was it to use the Auto Mode route 
visualization screen, including the ATC buttons?’’ The pilots responses were collected as described in a) above. 
Generally, the Evaluation Pilots found the features of the HMI display design relatively easy to use with average 
scores on the Likert scale all above 4. 
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Figure 14. Post-flight questionnaire items related to TAP understandability and usability. 
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Figure 15. Post-flight questionnaire items related to specific TAP HMI design changes. 
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c) General Feelings about TAP ’s Utility and Effect 

The Evaluation Pilots were asked to select from a list of 28 items those they consider to be the main benefits 
of TASAR and the TAP application. No limit was placed on the quantity they could select. Figure 16 presents the 
set of benefits selected by at least 50 percent of the eight independent Evaluation Pilots (excluding the TASAR team 
pilots), with the percentage of pilots selecting each benefit indicated. Most prevalent in this set were benefits directly 
related to the flight itself, including fuel and time savings, optimal routing, and airspace hazard avoidance. TASAR 
was also viewed by most Evaluation Pilots as providing equipage justification for ADS-B IN, EFB, and cockpit 
Internet access. Completing the set was workload reduction for pilots in assessing route-change impacts and 
workload reduction for air traffic controllers in flight optimization. Not selected by the majority were workload 
reductions for pilots and controllers in other areas (e.g., communication, coordination, maintaining situation 
awareness). Interestingly, no Evaluation Pilot identified dispatcher workload reduction in post-departure flight 
optimization as a main benefit, possibly indicating a gap in current operations that TASAR may be well -positioned 
to fill. 



Figure 16. Main benefits of TAP reported by at least 50% of the pilots. 


4. TAP Enhancement Recommendations 

The Evaluation Pilots emphasized the importance of the consistency of TAP’s findings, both between the two 
modes, and over time: “Solutions presented by TAP in Auto mode should be consistent between updates. Otherwise 
the pilots perceive it as inconsistent. ” These comments were elicited when TAP presented fluctuating optimizations 
over relatively short timespans. Based on detailed post-flight analysis, these apparent fluctuations were generally 
valid outcomes resulting from changing trajectory dynamics or new intruder traffic, making optimal routes 
temporarily unavailable. Nevertheless, the Evaluation Pilots were clearly more comfortable when TAP settled on a 
single solution for an extended period. Methods are being investigated in which new optimization searches are 
seeded with current solutions to minimize the occurrence of inconsequential solution fluctuations. 

The Evaluation Pilots also provided a wealth of additional suggestions for TAP HMI and functionality 
enhancements. The most common feedback concerned the Auto Mode “Selected Optimization” Screen. This screen 
displays a graphical representation of the lateral portion of the route change selected by the pilot, in addition to 
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regularly updated predictions of the route change and its fuel and time outcomes. Many of the Evaluation Pilots 
desired a more detailed depiction of the current route to assist in correlating TAP recommendations with other 
aircraft displays (e.g., such as navigation and weather radar displays). Some pilots requested greater use of the 
EFB’s touch-screen interface (e.g., for entering waypoints). Suggestions were also made regarding enhancing TAP 
functionality, including the incorporation of new airspace hazard classes into the optimization (e.g., volcanic ash, 
icing), and to account for preferred routings and ATC sector boundaries. 

D. TAP Operational Use Outcomes 

The TAP operational use outcomes are addressed under the follow headings: ADS-B traffic counts; 
optimization outcomes; computed time and fuel savings validation; trajectory geometry effects; and ATC handling. 

1. ADS-B Traffic Counts 

TAP processed 710 unique ADS-B targets (i.e., traffic aircraft) during 19.5 hours of data collection (which 
excluded the departure and arrival flight phases). Analysis of ground -recorded traffic data indicates this was 
approximately 12 percent of the total traffic (non- ADS-B and ADS-B) within ADS-B detection range. More ADS-B 
traffic was detected on northbound Flights 5 through 10, which traversed New York’s airspace, than on Flight 1 to 
the west, and Flights 3 and 4 to the south. Figure 17 illustrates traffic counts by flight (left) and the distribution of 
ADS-B detection range over the 19.5 hours (right). Flight 1 had a higher traffic count (67) than was expected due to 
the ownship detecting ADS-B traffic at Washington Dulles (IAD) and significant ADS-B traffic on north-south jet 
routes through Washington center (ZDC). About 85% of ADS-B traffic was first detected between 50 and 100 NM 
of the ownship. The ADS-B traffic aircraft generally cruised at higher altitudes than the ownship (FF320+) with a 
significant percentage of these ADS-B traffic aircraft first detected when either climbing or descending through the 
ownship altitude. These climbing and descending aircraft accounted for 12 of the 15 ADS-B traffic aircraft first 
detected less than 50 NM from the ownship. 



Figure 17: Total detected ADS-B targets by flight (left) and ADS-B detection range (right) 


2. Optimization Outcomes 

Figure 18 illustrates a sample optimization outcome from Flight 3 in relation to the original route and the SUA 
areas to be avoided, shown in Google Earth™. The red polygons represent SUA that TAP was designed to avoid. 
The green route indicates the original flight plan, and the yellow route indicates the “as flown” trajectory. The route 
was flown counter-clockwise. During the outbound leg to CRE and the initial inbound leg just past IFM, the 
Evaluation Pilot performed extensive testing of the TAP functions and HMI while the aircraft purposely remained 
on the planned route. Approximately halfway between IFM and FAY, the Evaluation Pilot consulted TAP for an 
optimization solution and made a route change request to ATC, receiving approval shortly before reaching FAY. 
The solution consisted of Direct FANHO (waypoint chosen by TAP from its navigation database). Direct CVI (the 
“rejoin” waypoint), bypassing FAY, RDU, and FVF from the original route. This solution passed just barely north 
of the Seymour Johnson Echo Military Operations Area, illustrating TAP’s function to avoid restricted airspace. 
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Figure 18. Sample operational outcome from Flight 3. 


Figure 19 shows a second example of a TAP Auto-mode trajectory optimization, from Flight 9 that was also 
proximate to an SUA (R-4001A). The orange route in the figure indicates TAP’s original solution, which was 
subsequently modified by ATC (yellow route) to provide extra separation from the airspace. An interesting aspect of 
both flight paths was that TAP’s avoidance solutions did not incorporate any buffers around the SUA. The TAP 
solutions of Flights 3 and 9 passed approximately 0.35 nm and 0.80 nm outside their respective SUAs, illustrating 
the accuracy of TAP’s algorithms. ATC’s modification of Flight 9’s requested route to reduce SUA proximity 
indicates a probable need to include SUA buffers in generating route-change requests. Additional buffering can be 
easily addressed in TAP calculations, though accounting for local ATC variations such as those encountered here 
would be more challenging. This unexpected observation serves as a good example of the value of flight-trial 
findings that might not have surfaced from a pure simulation experiment. 



Figure 19. Flight 9 Auto mode optimization. 
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Table 3 summarizes key parameters for all the TAP-inspired route optimizations requested of ATC during the 
flight trials. The Combo optimization comprised a combined lateral and vertical optimization. 


Table 3. Route optimization requests made to ATC and the outcome in time and fuel savings. 


Fit 

# 

AUTO 

MODE 3 

MANUAL 

MODE 3 

OPTIMIZATION 

TYPE 

ATC 

RESPONSE 

TIME SAVINGS 
(nuns.) 

TAP FMS 

FUEL SAVINGS (lbs) 
TAP FMS 

1 


(0) 

Lateral 

Approved 

3:21 

n/a b 

33 

n/a b 

3 

(1) 


Lateral 

Approved 

7:35 

15:00 

100 

150 

4 


(2) 

Lateral 

n/a c 

Invalid 11 

n/a 

Invalid 11 

n/a 

4 

(1) 


Combo 

Denied 

10:50 

- 

156 

- 

4 


(2) 

Lateral 

Deferred 

7:14 

- 

104 

- 

4 


(1) 

Lateral 

Approved 

n/a e 

12:00 

n/a e 

120 

5 

(1) 


Lateral 

Approved 

8:20 

8:00 

75 

127 

6 


(1) 

Lateral 

Approved 

2:58 

3:00 

28 

28 

7 


(1) 

Combo 

Approved 

13:31 

n/a b 

175 

n/a b 

7 


(1) 

Lateral 

Approved 

4:13 

4:00 

50 

70 

8 

ATC unable to address requests due to workload and/or frequency congestion 



9 


(2) 

Combo 

Denied 

- 

1:00 

- 

14 

9 


(1) 

Combo 

Approved 

n/a f 

4:00 

n/a f 

32 

10 


(1) 

Combo 

Approved 

1:26 

4:00 

14 

32 


d Numbers in parentheses indicate the number of off- route waypoints in the selected optimization. Zero indicates a 
direct routing to a downstream waypoint already on the flight-planned route. b FMS values not recorded. L ATC 
unable to respond due to workload. d Outlier TAP values (71:25 minutes, 1032 lbs. fuel savings). e No TAP 
solution ( hybrid of previous request). 1 Unable to compare TAP and FMS computed savings due to 
difference in request and ATC clearance. 

In total, 12 TAP-inspired ATC requests were made: two were denied, nine were approved, and ATC was unable 
to directly respond to the request in one case due to workload and/or frequency congestion. Some factors unique to 
the test design may have affected the ATC responses. These included the unusual round-robin nature of the flight 
paths, and the special treatment of the test aircraft by ATC discussed in subsections 4 and 5, respectively, below. 

3. Computed Time and Fuel Saving Validation 

Verification of the TAP time/fuel outcome predictions was not a specific flight test objective, but there was good 
agreement on three of the flights between the time and fuel savings computed by TAP and by the FMS. The 
discrepancies on the other flights are attributed to the lack of a high-fidelity Avanti performance model available to 
TAP for predicting ownship trajectories. It should be noted that the observed fuel and time savings recorded from 
the FMS were not dependent on TAP’s Avanti surrogate performance model, and represented real savings achieved 
by TAP. 

4. Trajectory Geometry Effects 

The nature of round-robin flights presented an optimization dilemma: how to optimize a flight whose destination 
is also the departure airport, albeit with intermediate trajectories of almost a thousand miles before returning to the 
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start point? In the case of flight plans five and six, the outbound and inbound legs also crossed each other and shared 
a common waypoint. While expedient from a planning perspective, these attributes caused some difficulties in 
TAP’s real-time modeling of the aircraft’s intended trajectory. This was compounded by the FMS design limitation 
of not outputting full route data on its GAMA A429 bus. The problem was partially mitigated by careful use of 
TAP’s optimization limit function, which defined the termination point for all optimization activity. Properly used, 
this allowed TAP to calculate sensible optimizations, even for flights returning to their start point. This problem was 
an artifact of the flight test design, and normal operational flights with differing origins and destinations would not 
encounter this issue. 

5. ATC Handling 

Although not a TAP issue, the nature of the ATC interactions with the flight -test aircraft on some of the flights 
limited the ability to fully test TAP capabilities. Prior to the test flights, the TASAR flight-trial test plan was 
coordinated with each ATC facility affected by the four intended routes. The purpose of the test was explained, as 
was the need to file non-standard and not-optimal routing and altitudes (i.e., to provide routes which TAP could 
optimize). Adjustments in some routings and altitudes were also made to accommodate local ATC restrictions, as 
requested. In most of the ATC facilities, bulletins were issued to the area supervisors each day that a test flight was 
planned through their airspace. Each evaluation flight included a task to request ATC approval of at least one TAP- 
derived optimization of the route. Given the unusual nature of the flight plan (i.e., round-robin and using non- 
standard routing), two unintended situations were occasionally experienced. The first was an eagerness by the 
controllers to accommodate the special needs of this pre-briefed NASA flight. In one case, ATC was very willing to 
accommodate a TAP request, and relayed it across several sector boundaries, causing a significant delay in request 
approval. The downstream controller eventually cleared the flight to the optimization fix, but by this time the chosen 
waypoint was passing abeam the aircraft, and it was impractical to complete the original optimization. It is unlikely 
that routine flights would have been treated in this privileged and somewhat unconventional manner. The second, 
less frequent, eventuality was an inability to accommodate any “special” requests due to ATC workload and 
frequency congestion. These issues will be addressed in future flights by avoiding round-robin flight plans and 
eliminating the need to coordinate the special routing in advance. 

IV. TAP Future Plans 

This first TAP flight trial provided an invaluable stepping-stone towards an operational deployment of the 
system using target customer equipment. A follow-on flight trial is in the planning stage for the late 2014 / early 
2015 timeframe, in preparation for operational -use testing of TASAR with one or two Part 121 partner air carriers 
later that year. The flight trial will endeavor to match the equipment and operations of the partner airlines as closely 
as possible, following the model established in the 2013 trials. This will involve some additional test-bed aircraft 
modifications, an updated TAP HMI, and porting of the TAP software into the new target environments. With the 
basic operation and avionics communications already established, there will be an increased focus on new user- 
driven TAP functionalities and further HMI enhancements. At the successful conclusion of these upcoming trials, 
the TAP system should have reached TRL 8 (mission qualified), and be ready for operational deployment to 
industry. 


V. Conclusion 

The TAP flight evaluations met or exceeded all of the objectives set for the program. TAP was found to operate 
successfully using the data available from commercially available ADS-B and broadband equipment. The software 
loaded and executed normally, and pilot usability and workload feedback was positive. The software generated 
usable lateral and vertical optimizations that were approved by ATC 75 percent of the time, and resulted in actual 
fuel and flight-time savings. These accomplishments were demonstrated in a TRL 6 relevant, semi-operational, 
environment with software that was successfully ported from a TRL 4 simulation platform. In the process, a number 
of valuable lessons were learned about the software, and the majority of the practical challenges to TAP’s 
deployment were overcome. Future research will build on this important foundation and is expected to lead to 
operational deployment of TAP on the revenue flights of one or two partner airlines. 


22 

American Institute of Aeronautics and Astronautics 



Acknowledgments 


This project was funded by NASA under contract NNL12AA06C. Development of the TAP software system 
was performed by Engility Corporation. 


References 


’Federal Aviation Administration, "NextGen Implementation Plan," Federal Aviation Administration; 2013, Washington, 
DC, 2013. 

2 Ballin, M., & Wing, D. (2012). Traffic Aware Strategic Aircrew Requests (TASAR). American Institute of Aeronautics and 
Astronautics, AIAA-2012-5623. doi:doi: 10.25 14/6.2012-5623. 

3 Woods, S.E., Vivona, R.A., Roscoe, D.A.. LeFebvre, B.C., Wing. D.J., & Ballin, M.G.. A Cockpit-based Application for 
Traffic Aware Trajectory Optimization. AIAA Guidance, Navigation, and Control Conference, Boston, MA, 2013. 

4 Henderson, J., Idris, H., & Wing, D. (2012). Preliminary Benefits Assessment of Traffic Aware Strategic Aircrew Requests 
(TASAR). American Institute of Aeronautics and Astronautics, AIAA-2012-5684. doi:doi: 10.25 14/6.2012-5684. 

5 Koczo, S., & Wing, D. (2013). An Operational Safety and Certification Assessment of a TASAR EFB Application. Paper 
presented at the Digital Avionics Systems Conference (DASC), 2013 IEEE/AIAA 32nd, 2A1-1-2A1-18. 

doi:10.1109/DASC.2013.6712530. 

6 Schnell T.. Cover M., Engler J.. Koczo S. (2014). Traffic Aware Strategic Aircrew Requests (TASAR) Human-in-the-Loop 
Assessment. NASA technical report No. NASA/CR-2014-000000. Manuscript submitted for publication. 

7 Karr, D., Vivona, R., Roscoe, D., Depascale, S., & Wing, D. (2012). Autonomous Operations Planner: A Flexible Platform 
for Research in Flight-Deck Support for Airborne Self-Separation. American Institute of Aeronautics and Astronautics, AIAA- 
2012-5417. doi:doi: 10.25 14/6.201 2-541 7. 

8 Federal Aviation Administration, "Guidelines for the Certification, Airworthiness, and Operational Use of Portable 
Electronic Flight Bags," AFS-400, AC 120-76B, Washington, DC, 2012. 

9 Henderson, J. (2013). Traffic Aware Strategic Aircrew Requests (TASAR) Concept of Operations. NASA technical report 
No. NASA/CR-2013-218001. Hampton, VA: NASA Langley Research Center. 

I0 Koczo, S., "Analysis of Operational Hazards and Safety Requirements for Traffic Aware Strategic Aircrew Requests 
(TASAR)," NASA technical report No. NASA/CR-20 13-2 18002, 2013. 

"Federal Aviation Administration, "System Safety Analysis and Assessment for Part 23 Airplanes." Federal Aviation 
Administration, AC 23.1309-1E, Washington, DC, 2011. 

i2 RTCA SC-135, "Environmental Conditions and Test Procedures for Airborne Equipment," RTCA, Inc., RTCA/DO-160G, 
Washington, D.C., 2010. 

’’RTCA SC-205, "DO-178C Software Consideration in Airborne Systems and Equipment Certification," RTCA, Inc., 
Washington, D.C., 2011. 

14 Wing, D., Ballin, M., Koczo, S., "Developing an Onboard Traffic-Aware Flight Optimization Capability for Near-Term 
Low-Cost Implementation," Aviation 2013, AIAA-20 13-4231, AIAA, Washington, DC, 2013. 

"Roscoe, A., and Ellis, G., "A Subjective Rating Scale for Assessing Pilot Workload in Flight: A Decade of Practical Use," 
Royal Aerospace Establishment, ADA227864, Farnborough, U.K., 1990. 

"Federal Aviation Administration, "Guidelines for the Certification, Airworthiness, and Operational Use of Portable 
Electronic Flight Bags," AC 120-76B, Washington, DC, 2012. 

"ARINC, "828-3 Electronic Flight Bag (EFB) Standard Interface." ARINC Incorporated, 828-3, Annapolis MD, 2012. 

1S ARINC, "834-3 Aircraft Data Interface Function (ADIF)," ARINC Incorporated, 828-3, Annapolis MD, 2012. 

19 Brooke, J. (1996). In Jordan P. (Ed.), SUS: A Quick and Dirty Usability Scale. Bristol, PA: Taylor & Francis Inc. 


American Institute of Aeronautics and Astronautics 



